The time resolution of photon detection systems is important for a wide range of applications in physics and chemistry. It impacts the quality of time-resolved spectroscopy of ultrafast processes and has a direct influence on the best achievable time resolution of time-of-flight detectors in high-energy and medical physics. For the characterization of photon detectors, it is important to measure their exact timing properties in dependence of the photon flux and the operational parameters of the photodetector and its accompanying electronics. We report on the timing of silicon photomultipliers (SiPM) as a function of their bias voltage, electronics threshold settings and the number of impinging photons. We used ultrashort laser pulses at 400 nm wavelength with pulse duration below 200 fs. We focus our studies on different types of SiPMs (Hamamatsu MPPC S10931-025P, S10931-050P and S10931-100P) with different SPAD sizes (25μm, 50μm and 100μm) coupled to the ultrafast discriminator amplifier NINO. For the SiPMs, an optimum in the time resolution regarding bias and threshold settings can be reached. For the 50μm type, we achieve a single photon time resolution of 80 ps sigma, and for saturating photon fluxes better than 10 ps sigma.